Digital non-linear editing (DNLE) is a process by which digital media may be edited. DNLE, as the name implies, is performed on digital media stored as data in digital media files on a digital random access medium. DNLE may be conducted in a non-linear fashion because the digital media files in which the digital media is stored can be randomly accessed. Thus an editor may access a piece of the digital media without having to proceed sequentially through other pieces of the digital media stored in the same or other digital media files. More than one editor also may be able to access different pieces of the same digital media contemporaneously. The digital media may be a digitized version of a film or videotape or digital media produced through live capture onto a disk of a graphics or animation software application. Example commercial DNLE systems include the Media Composer(copyright) or the Avid(copyright) Symphony(trademark) video production system or NewsCutter(copyright) news editing system available from Avid Technology, Inc. For a more detailed description of DNLE, see Digital Nonlinear Editing, New Approaches to Editing Film and Video, Second Edition, 1998 by Thomas Ohanian.
Basic components of digital media are segments, which are described in more detail below. For audio segments, each segment includes a plurality of sequential audio samples. For a video, a segment comprises at least one digital image, where each digital image comprises an array of picture elements called pixels. A video segment has several properties or attributes associated with digital images of the video segment. The attributes include, inter alia, image formatting attributes, color attributes, storage location attributes, and source attributes. Source attributes define the source data structures from which a segment originates. Segments and their source data structures are described below in more detail.
Formatting attributes may include: the color format of the pixels; the pixel depth of the pixels; the number of rows and columns of pixels, commonly referred to as the image size; the shape of the pixels, for example, square and non-square; the type of video signal on which the digital image is, was or will be transmitted; the video rate at which the signal is, was or will be transmitted; the sampling rate by which the video signal is, was or will be transmitted, for example, 4:2:2 or 4:4:4; and the pixel clock rate used to transfer the digital pixel data within a DNLE system.
Many of these formatting attributes are defined by standards. For example, in a standard square pixel NTSC image format, there are 640 columns and 480 rows of pixels. Further, in both non-square pixel PAL format and non-square NTSC format, pixel information is transmitted at a rate of 27 MHZ.
Video signals may be classified into two major types: composite signals and component signals. Component signals represent a pixel with multiple color components, each component defining a value along a dimension of the color space in which the color being represented is defined. A composite video signal is an analog signal that uses a high frequency subcarrier to encode color information. The subcarrier is a sinewave of which the amplitude is modulated by the saturation of the color represented by the signal, and the hue of the color is encoded as a phase difference from a color burst. Analog composite signals generally are used to broadcast television video signals.
The pixel depth of an image defines the number of binary digits necessary to store the color of a pixel on a computer-readable medium. For example, a common pixel depth for a component color format is 24 bits, where 8 bits are used to store each color component.
Video and audio segments may also have modification attributes associated with them. Modification attributes of a segment are attributes associated with modifications made to the segment by a system user such as an editor or a colorist. These modifications can be applied to digital images and audio samples on a DNLE. Video or image modifications include, inter alia, resizing an image, keying, panning and scanning, blending, box wipes, dissolves, color effects, color correction, and other various modifications and effects known to those skilled in DNLE. Audio modification include, inter alia, mixing, sweetening, manipulating audio waveforms, and other various modifications and effects known to those skilled in the art.
A sub-class of video modification is color modification. Color modification may be performed both to correct color errors due to process errors and to adjust the colors used in the video for artistic expression. For a given image, color modifications may be applied to all pixels in the image or pixels of a portion of the image. Color modifications may include enhancing contrasts or color in an image to give a program an overall xe2x80x9clook,xe2x80x9d or applying special effects to selected segments. Other color modifications may be made by an editor during an editing session to correct problems with color or lighting resulting from the source of the media. Such corrections may include color balancing for camera and lighting differences, correcting for film processing differences, matching colors and tones from shot to shot, or adjusting video levels for differences in source tapes, source decks, etc.
There are a variety of component color formats used to represent color. Component color formats are used in component video signals as well as in digital pixel streams. An digital image storage RGB (Red, Green, Blue) format represents a color with a red component, a green component and a blue component. CMY (Cyan, Magenta, Yellow, Black) format represents a color with a cyan component, a magenta component, and a yellow component. CMY is a format commonly used by printers. The CMY components are color opposites of RGB components. In a three-dimensional coordinate system, each component of either the RGB or the CMY format represents a value along an axis, the combination of the values defining a cubic color space.
The color formats HSL (Hue, Saturation, Lightness or Luminance) and HSV (Hue, Saturation, Value) represent a color with a hue component, a saturation component, and a luma component. In a three-dimensional coordinate system, the luma component represents a value along a luma axis, the hue component represents the angle of a chroma vector with respect to the luma axis and the saturation component represents the magnitude of the chroma vector. The combination of the values defines a hexagonal cone-shaped color space around the luma axis.
YCrCb, YUV, and YIQ are three color formats that represent a color with a luma component Y, and two chroma components, Cr and Cb, U and V, or I and Q, respectively, that define a chroma vector. In a three-dimensional coordinate system, each component of either the YCrCb, YUV, and YIQ format represents a value along an axis, the combination of the values defining a cylindrical color space around the luma axis. The chroma components define the chroma vector. In data formats with a luma component, the luma component can be used independently to represent a pixel in a black and white image to be displayed, for example, with a black and white monitor.
A typical color modification in HSL color space may include increasing a color component or a combination of color components for all pixels in each digital image of a section of digital media. Typically, an editor accesses a segment of a composition that represents the section of media through an editor interface and inputs desired color modifications through the editor interface. Some systems permit an editor to apply color modifications to only portions of a digital image. Portions of a digital image can also be described as one or more pixels. For example, an editor may select with a mouse, keyboard, or some other editor input device a portion of the image and define color modifications for the selected portion. A suitable commercial system for color modification is Avid Media Illusion(trademark) available from Avid Technology, Inc. The Avid Media Illusion Reference Guide, available from Avid Technology, Inc. is herein incorporated by reference. Other commercial software applications may be used.
One problem with current techniques for modifying digital media is that a single modification generally cannot be specified for several segments in a composition sharing a common attribute. An editor generally has to access every segment sharing the common attribute to make the modification. This process can be time-consuming and prone to error. An editor may intend on making the same modification to a first and second segment sharing a common attribute, but the modifications may be inconsistent. Further, different editors may be working on the different segments sharing a common attribute. Although copying a modification from one segment to the other is one solution to this inconsistency, the other segment is not always available, and having to copy adds more time to the editing process. As the number of segments to be modified increases, time is added to the editing process.
The system and method described herein permits a modification to be specified for several segments sharing a common attribute. Such shared modification may be combined with other separately specified modifications made to a composition or part of a composition. Such modification may be audio modifications specified for audio segments or a video modification specified for video segments.
Accordingly, in one embodiment, a method of processing a modification to be applied to sections of digital media on a digital nonlinear editing system is provided, where a first segment of a media composition represents at least a section of a digital media source. An indication of a modification to be applied to the first segment is received, and a first attribute of the first segment is identified. A first indication of the modification is stored such that the modification is applied to other segments on the system having the first attribute in common with the first segment.
In an aspect of this embodiment, a common attribute parameter of the first segment to be applied to determine a common attribute that the first segment and other segments on the system share in common is accessed. The first attribute is identified in accordance with the common attribute parameter.
In another aspect of this embodiment, a first data structure corresponding to the first attribute is identified, where the first data structure stores modifications made to segments having the first attribute in common with the first segment. The modification is stored in the first data structure.
In yet another aspect of this embodiment, a first table corresponding to a digital media composition to which first the first segment belongs is maintained. The first table includes a plurality of entries, and each entry contains a segment attribute and an indication of a modification to be applied to segments of the media composition having the segment attribute. The first indication and the first attribute are stored in a first entry of the first table.
In another aspect of this embodiment, the first attribute is a name of a source data structure from which the first segment originates.
In another aspect of this embodiment, the first segment includes a first identifier that identifies the first data structure, and first identifier of the first segment is accessed to determine the first data structure.
In yet another aspect of this embodiment, segments on the system are sequentially accessed, where the first indication of the modification is stored in each accessed segment that has the first attribute in common with the first segment.
In another aspect of this embodiment, the first data structure is a first table including a plurality of entries, and each entry includes a modification to be applied to segments having the first attribute in common.
In another embodiment, a method of applying a modification to a first section of a digital media source on a digital nonlinear editing system is provided, where a first segment of a media composition represents the first section. The first segment is accessed, and a first source data structure from which the first segment originates is accessed, where the first source data structure represents at least the first section of the digital media source. It is determined whether the first source data structure includes a first indication that defines a first modification to be applied to sections of the digital media source represented by segments that originate from the first source data structure. If the first source data structure includes the first indication, the first modification is applied to the first section of media represented by the first segment.
In an aspect of this embodiment, a second source data structure from which the first source data structure originates is identified, the second source data structure representing the digital media source. It is determined whether the second source data structure includes a second indication that defines a second modification to be applied to sections of the digital media represented by segments that originate from the second source data structure. If the second source data structure includes the second indication and the first source data structure does not include the first indication, the second modification is applied to the first section of digital media source.
In yet another aspect of this embodiment, it is determined whether the first segment includes a second indication that defines a second modification defined by the composition to be applied to the first section of the digital media source represented by the first segment. If the first segment includes the second indication, the second modification is applied to the first section of the digital media source, such that, if the first source data structure includes the first indication, a combined modification of the first and second modification is applied to the first section of the digital media.
In another embodiment, a method of applying a modification to a first section of a digital media source on a digital nonlinear editing system is provided, where a first segment of a media composition represents the first section. The first segment is accessed and each entry of a first table that corresponds to the media composition is accessed. The first table includes a plurality of entries, and each entry contains a segment attribute and an indication of a modification to be applied to segments of the composition having the segment attribute. For each entry, it is determined whether the first segment has the segment attribute, and if the segment has the segment attribute, modifying the first section of digital media with the image modification contained in the entry.
In an aspect of this embodiment, it is determined whether the first segment includes an indication of a modification defined specifically for the composition and to be applied to the first section of the digital media source represented by the first segment. If the first segment includes the indication defined specifically for the composition, the modification defined specifically for the composition is applied to the first section of the media source, in addition to the modifications applied from the first table.
In another aspect of this embodiment, for each entry of the first table for which the first segment does have the attribute, an instruction from a user is received indicating whether to modify the first section of digital media with the modification of the entry. If the instruction instructs to modify, the first section of digital media is modified with the modification of the entry.